Reinforcement learning allows machines to learn from their own experience. Nowadays, it is used in safety-critical applications, such as autonomous driving, despite being vulnerable to attacks carefully crafted to either prevent that the reinforcement learning algorithm learns an effective and reliable policy, or to induce the trained agent to make a wrong decision. The literature about the security of reinforcement learning is rapidly growing, and some surveys have been proposed to shed light on this field. However, their categorizations are insufficient for choosing an appropriate defense given the kind of system at hand. In our survey, we do not only overcome this limitation by considering a different perspective, but we also discuss the applicability of state-of-the-art attacks and defenses when reinforcement learning algorithms are used in the context of autonomous driving.

在线系统缺乏连续性的最常见原因之一是源自广泛流行的网络攻击，称为分布式拒绝服务（DDOS），在该网络攻击中，受感染设备（僵尸网络）网络被利用以通过淹没服务的计算能力。攻击者的命令。这种攻击是通过通过域名系统（DNS）技术通过域生成算法（DGAS）来进行的，这是一种隐身连接策略，但仍留下可疑的数据模式。为了发现这种威胁，已经取得了分析的进步。对于大多数人来说，他们发现机器学习（ML）是一种解决方案，可以在分析和分类大量数据方面非常有效。尽管表现出色，但ML模型在决策过程中具有一定程度的晦涩难懂。为了解决这个问题，ML的一个被称为可解释的ML的分支试图分解分类器的黑盒性质，并使它们可解释和可读。这项工作解决了在僵尸网络和DGA检测背景下可解释的ML的问题，我们最了解的是，当设计用于僵尸网络/DGA检测时，第一个具体分解了ML分类器的决定，因此提供了全球和本地。解释。

尽管机器学习容易受到对抗性示例的影响，但它仍然缺乏在不同应用程序上下文中评估其安全性的系统过程和工具。在本文中，我们讨论了如何使用实际攻击来开发机器学习的自动化和可扩展的安全性评估，并在Windows恶意软件检测中报告了用例。

在过去的几年中，对针对基于学习的对象探测器的对抗性攻击进行了广泛的研究。提出的大多数攻击都针对模型的完整性（即导致模型做出了错误的预测），而针对模型可用性的对抗性攻击，这是安全关键领域（例如自动驾驶）的关键方面，尚未探索。机器学习研究社区。在本文中，我们提出了一种新颖的攻击，对端到端对象检测管道的决策潜伏期产生负面影响。我们制作了一种通用的对抗扰动（UAP），该扰动（UAP）针对了许多对象检测器管道中的广泛使用的技术 - 非最大抑制（NMS）。我们的实验证明了拟议的UAP通过添加“幻影”对象来增加单个帧的处理时间的能力，该对象在保留原始对象的检测时（允许攻击时间更长的时间内未检测到）。

评估机器学习模型对对抗性示例的鲁棒性是一个具有挑战性的问题。已经证明，许多防御能力通过导致基于梯度的攻击失败，从而提供了一种错误的鲁棒感，并且在更严格的评估下它们已被打破。尽管已经提出了指南和最佳实践来改善当前的对抗性鲁棒性评估，但缺乏自动测试和调试工具，使以系统的方式应用这些建议变得困难。在这项工作中，我们通过以下方式克服了这些局限性：（i）根据它们如何影响基于梯度的攻击的优化对攻击失败进行分类，同时还揭示了两种影响许多流行攻击实施和过去评估的新型故障； （ii）提出了六个新的失败指标，以自动检测到攻击优化过程中这种失败的存在； （iii）建议采用系统协议来应用相应的修复程序。我们广泛的实验分析涉及3个不同的应用域中的15多个模型，表明我们的失败指标可用于调试和改善当前的对抗性鲁棒性评估，从而为自动化和系统化它们提供了第一步。我们的开源代码可在以下网址获得：https：//github.com/pralab/indicatorsofattackfailure。

Graph Neural Networks (GNNs) achieve state-of-the-art performance on graph-structured data across numerous domains. Their underlying ability to represent nodes as summaries of their vicinities has proven effective for homophilous graphs in particular, in which same-type nodes tend to connect. On heterophilous graphs, in which different-type nodes are likely connected, GNNs perform less consistently, as neighborhood information might be less representative or even misleading. On the other hand, GNN performance is not inferior on all heterophilous graphs, and there is a lack of understanding of what other graph properties affect GNN performance. In this work, we highlight the limitations of the widely used homophily ratio and the recent Cross-Class Neighborhood Similarity (CCNS) metric in estimating GNN performance. To overcome these limitations, we introduce 2-hop Neighbor Class Similarity (2NCS), a new quantitative graph structural property that correlates with GNN performance more strongly and consistently than alternative metrics. 2NCS considers two-hop neighborhoods as a theoretically derived consequence of the two-step label propagation process governing GCN's training-inference process. Experiments on one synthetic and eight real-world graph datasets confirm consistent improvements over existing metrics in estimating the accuracy of GCN- and GAT-based architectures on the node classification task.

Neuromorphic systems require user-friendly software to support the design and optimization of experiments. In this work, we address this need by presenting our development of a machine learning-based modeling framework for the BrainScaleS-2 neuromorphic system. This work represents an improvement over previous efforts, which either focused on the matrix-multiplication mode of BrainScaleS-2 or lacked full automation. Our framework, called hxtorch.snn, enables the hardware-in-the-loop training of spiking neural networks within PyTorch, including support for auto differentiation in a fully-automated hardware experiment workflow. In addition, hxtorch.snn facilitates seamless transitions between emulating on hardware and simulating in software. We demonstrate the capabilities of hxtorch.snn on a classification task using the Yin-Yang dataset employing a gradient-based approach with surrogate gradients and densely sampled membrane observations from the BrainScaleS-2 hardware system.

Generalisation to unseen contexts remains a challenge for embodied navigation agents. In the context of semantic audio-visual navigation (SAVi) tasks, the notion of generalisation should include both generalising to unseen indoor visual scenes as well as generalising to unheard sounding objects. However, previous SAVi task definitions do not include evaluation conditions on truly novel sounding objects, resorting instead to evaluating agents on unheard sound clips of known objects; meanwhile, previous SAVi methods do not include explicit mechanisms for incorporating domain knowledge about object and region semantics. These weaknesses limit the development and assessment of models' abilities to generalise their learned experience. In this work, we introduce the use of knowledge-driven scene priors in the semantic audio-visual embodied navigation task: we combine semantic information from our novel knowledge graph that encodes object-region relations, spatial knowledge from dual Graph Encoder Networks, and background knowledge from a series of pre-training tasks -- all within a reinforcement learning framework for audio-visual navigation. We also define a new audio-visual navigation sub-task, where agents are evaluated on novel sounding objects, as opposed to unheard clips of known objects. We show improvements over strong baselines in generalisation to unseen regions and novel sounding objects, within the Habitat-Matterport3D simulation environment, under the SoundSpaces task.

Multi-document summarization (MDS) has traditionally been studied assuming a set of ground-truth topic-related input documents is provided. In practice, the input document set is unlikely to be available a priori and would need to be retrieved based on an information need, a setting we call open-domain MDS. We experiment with current state-of-the-art retrieval and summarization models on several popular MDS datasets extended to the open-domain setting. We find that existing summarizers suffer large reductions in performance when applied as-is to this more realistic task, though training summarizers with retrieved inputs can reduce their sensitivity retrieval errors. To further probe these findings, we conduct perturbation experiments on summarizer inputs to study the impact of different types of document retrieval errors. Based on our results, we provide practical guidelines to help facilitate a shift to open-domain MDS. We release our code and experimental results alongside all data or model artifacts created during our investigation.

We consider the problem of two active particles in 2D complex flows with the multi-objective goals of minimizing both the dispersion rate and the energy consumption of the pair. We approach the problem by means of Multi Objective Reinforcement Learning (MORL), combining scalarization techniques together with a Q-learning algorithm, for Lagrangian drifters that have variable swimming velocity. We show that MORL is able to find a set of trade-off solutions forming an optimal Pareto frontier. As a benchmark, we show that a set of heuristic strategies are dominated by the MORL solutions. We consider the situation in which the agents cannot update their control variables continuously, but only after a discrete (decision) time, $\tau$. We show that there is a range of decision times, in between the Lyapunov time and the continuous updating limit, where Reinforcement Learning finds strategies that significantly improve over heuristics. In particular, we discuss how large decision times require enhanced knowledge of the flow, whereas for smaller $\tau$ all a priori heuristic strategies become Pareto optimal.